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SYSTEM AND METHOD FOR SELECTION 
OF A VOICE USER INTERFACE DIALOGUE 

RELATED APPLICATION 

This application is a continuation-in-part from U.S. 
Patent Application serial no. 10/263,552, filed 
October 3, 2002, and entitled, "Dynamic and Adaptable 
5 System and Method for Selecting a User Interface Dialogue 
Model" now U.S. Patent No. . 



TECHNICAL FIELD OF THE INVENTION 

The present invention relates in general to 
10 communication systems and more particularly to an 
adaptable system and method for selecting a user 
interface dialogue model . 
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BACKGROUND OF THE INVENTION 

Customers contact organizations such as 
telecommunications companies in order to accomplish tasks 
including ordering a service, requesting repair service, 
5 or asking questions about their bill. When a customer 
contacts the organization, the customer's call is 
connected to a user interface which may be a customer 
service representative or any of a number of automated 
systems. Currently, when designing a system for 

10 communicating with customers, the determination of the 

type interface or dialogue model a customer will interact 
with is decided at an early stage within the design of 
the communication system. Typically, designers choose a 
primary interface, such as a speech directed dialogue 

15 interface, and a secondary or back-up interface, such as 
a touch tone interface, in case the primary interface 
fails . 

As a communication system changes over time, the 
existing dialogue model is often ill suited for newly 

20 added tasks. However, the existing dialogue model is 
often maintained because redesigning the system to 
incorporate a new dialogue model is prohibitively 
expensive. This may lead to a decrease in the 
effectiveness of the system as well as customer 

25 dissatisfaction and frustration. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

A more complete understanding of the present 
embodiments and advantages thereof may be acquired by 
referring to the following description taken in 
5 conjunction with the accompanying drawings, in which like 
reference numbers indicate like features, and wherein: 
FIGURE 1 is a depiction of a communication system 
including a selection module, user interface dialogue 
module library and selection criteria library according 
10 to the teachings of the present invention; 

FIGURE 2 shows a flow diagram of a user interface 
selection system according to the present invention; 

FIGURE 3 is a flow diagram of a method for selecting 
user interface dialogue models; 
15 FIGURE 4 is a flow diagram of a user interface 

selection system including a criteria scoring matrix 
according to teachings of the present disclosure; and 

FIGURE 5 is a flow diagram showing a diagram of a 
dialogue selection table according to teachings of the 
2 0 present invention. 
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DETAILED DESCRIPTION OF THE INVENTION 

Preferred embodiments and their advantages are best 
understood by reference to FIGURES 1 through 5, wherein 
like numbers are used to indicate like and corresponding 
5 parts. 

Now, referring to FIGURE 1, a communication system 
for communicating with customers, depicted generally at 
10, is shown. Communication system 10 is preferably 
designed to receive communications from customers or 

10 other constituents of an organization. In the present 
embodiment, communication system. 10 is designed to 
receive incoming communications from telecommunications 
customers on behalf of a telecommunications company. 

Communication system 10 includes selection module 12 

15 which is operably connected with both user interface 
dialogue model 14 and selection criteria library 16. 
Persona library 50 as well as a plurality of 
communication task modules such as module A, 6 0 module B, 
70 and module C, 80 are also operably connected with 

20 selection module 12. 

In the present embodiment, selection module 12 
operates to select an appropriate user interface for a 
particular node of a communication task. Selection 
module 12 includes criteria evaluation module 18, 

25 interface scoring module 20, interface selection module 
22, persona evaluation module 24 and feedback module 25. 
Selection module 12 is generally operable to communicate 
with selection criteria library 16 and user interface 
dialogue model library 14. User interface dialogue model 
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library 14 is a database or other information storage 
means for storing information about a plurality of 
different user interface dialogue models 3 0 which may 
also be referred to as user interfaces or dialogue 
5 models. User interface dialogue model library 14 

includes criteria scores 38 for each user interface type. 

In the present embodiment, user interface dialogue 
model library 14 includes four user interface types: 
service representative 31, touch tone IVR 32, speech 

10 directed dialogue 34, and speech statistical language 
model/natural language understanding (SLM/NLU) 36. In 
the present embodiment, criteria scores 38 are listed for 
each individual user interface type and are also 
associated with a criteria number 39. Each criteria 

15 number 39 corresponds to a criteria 40 listed within 
selection criteria library 16. Note that the term 
"criteria" as used herein may be used to refer to a 
single criterion or multiple criteria. 

Selection criteria library 16 is a database or other 

2 0 information storage means for storing information related 
to selection criteria for selecting a user interface 
type. Selection criteria library 16 includes a listing 
of a plurality of criteria 40 and associated criteria 
weight scores 42. Criteria 40 may include, for example, 

25 criteria such as cost, customer satisfaction, automation 
rate, task completion rate, task complexity, confidence 
of outcome, length of time in system, list length, 
interface used in previous state, and a dialogue state 
counter. These example criteria 40 are described in 

30 greater detail with respect to FIGURE 2, below. Criteria 
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weight 42 is an assigned valuation of the importance or 
weight given to each particular criteria 40. In the 
present embodiment, criteria weight 42 is dynamic and 
adaptable. Criteria weight 42 may be adapted, modified 
5 or changed to reflect changing environmental factors, 
changing customer factors, and changing organizational 
factors . 

As an example, environmental factors may include 
background noise and cell phone (modality) use. High 

10 background noise may shift the VUI selection to one that 
operates better in high noise. Cell phone may shift the 
VUI selection to one that does not require keying 
information manually, but rather utilizes voice commands. 
Examples of customer factors include changing customer 

15 views and priorities generally and system-specific 

factors. General views and priorities may include, for 
instance, customer views on cost and customer service 
which may be gained from survey data or other similar 
data. System-specific trigger events such as increasing 

2 0 the weight of the "time in system" criteria if the 

average time a customer spends in the system exceeds a 
pre-selected average. An example of a changing 
organization factor may be that an organization may 
periodically change its goals with respect to factors 
25 such as cost and automation rate. In these instances, 

the criteria weights may be changed to reflect changes in 
an organization's goals or priorities. 

Selection module 12 operates to select a proper user 
interface type from user interface dialogue model library 

3 0 14 for anyone of the plurality of communication task 
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modules 60, 70, or 80. Each communication task module 
such as communication task module A 60 includes a number 
of steps or nodes 62 required to complete the task. Each 
node may have associated therewith one or more variables 
5 64. Variables 64 may be used to adjust the value of 
criteria weight 42 or criteria score 39 to reflect 
node-specific factors. One example of a node-specific 
factor that is used to adjust criteria weight 42 is list 
length. An example of where variable 64 may be used to 

10 adjust a criteria score is with respect to the time in 

system score related to a service representative 31 form 
of user interface because the time in system for such an 
interface will be directly affected by whether or not a 
customer service representative is available or whether 

15 the customer will be put on hold. 

In operation, selection module 12 evaluates which 
user interface type 30 to use a particular communication 
task such as communication task module A 60. Selection 
module 12 first evaluates which user interface type 30 to 

2 0 use for the first node 62 or step within communication 
task module A 60. After successful completion of each 
node 62, selection module 12 evaluates and selects a user 
interface type for each successive node 62 within the 
communication task module. Criteria evaluation module 18 

25 begins this process by retrieving criteria 40 and their 
associated criteria weights 42 from selection criteria 
library 16. Interface scoring module 20 then retrieves 
criteria scores 38 for each of the plurality of listed 
user interface dialogue models 30. Interface score 

30 module 20 then, for each user interface dialogue model 



AUS01:320843.2 



ATTORNEY DOCKET 
064198 . 0162 
(T00496) 



PATENT APPLICATION 



8 

30, multiplies each criteria score 38 with each 
corresponding criteria weight 42 and adds the resulting 
values to determine an interface score. Interface 
selection module 22 then selects the user interface 
5 dialogue model that has the highest interface score. The 
communication node is then directed to the selected user 
interface dialogue model and proceeds until the 
completion of the current node of the communication task. 
In some embodiments, interface scoring module 2 0 may 

10 incorporate one or more values such as variable value 65 
associated with a node, such as node 66. Variable value 
65 may then be factored into a criteria weight value 42 
or a criteria score value 39. For example, interface 
scoring module 20 may multiply selected criteria weight 

15 67 by variable 65. By multiplying the variable value 
with criteria weight 67, interface selection module 22 
adjusts the criteria weight for node specific variables. 
As another example, interface scoring module 2 0 may 
incorporate variable value 69 as the criteria score for 

20 service representative 30 user interface type dialogue 
model. In this particular instance, variable 69 is a 
factor that represents the average time (using a moving 
average) in system for a call routed to a service 
representative. Accordingly, when there are not enough 

25 service representatives to handle incoming calls, and 

customers must be put on hold before they can speak to a 
customer service representative, this may be factored 
into the criteria score. 

After selection module 12 selects a user interface 

3 0 type, persona evaluation module 24 may evaluate whether 
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the selected user interface dialogue is appropriate for 
the particular call based on call-specific aspects known 
as persona. In particular, persona evaluation module 24 
may be used to evaluate whether or not a specific call is 
5 appropriate for user interface types that include a voice 
recognition component such as a speech directed dialogue 
34 or SLM/NLU 36. Persona library 50 contains call 
specific data, including data related to the caller's 
choice of words, the caller's pitch, modulation, and 

10 attitude. If persona evaluation module 24 determines 

that the selected user interface type is not appropriate 
for a particular call then persona evaluation module 
communicates with interface selection module 22 to select 
the user interface type with the next highest interface 

15 score. The process of persona evaluation 24 may then be 
repeated for the second selected user interface type. 

Selection module 12 also includes feedback module 
25. Feedback module 25 automatically provides feedback 
to the variables 64 associated with the communication 

20 task module 60, criteria weights 42, and criteria scores 
38. For example, feedback module 25 operates to and keep 
track of the success rate and time lengths associated 
with the plurality of user interface types. Feedback 
module 25 may automatically adjust criteria weights 42, 

25 communication task variables 64 and criteria scores 38 to 
reflect the current state of the communication system 10. 
Feedback module 25 also allows a system administrator to 
manually adjust the criteria weight 42 and criteria 
scores 3 9 to reflect changing environmental factors, 

30 customer factors, and organizational factors. 
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For example, if the customer begins with a very 
broad topic, the speech SLM/NLU 36 style of "How May I 
help You/Speak Freely" would be utilized. However, if 
the environment has a lot of background noise, the Touch 
5 Tone 32 style would be utilized through operation of 

persona evaluation module 24. As the customer continues 
their interaction, the dialogue model could change from 
Speech Directed Dialogue 34 to a Service Representative 
31. An important aspect of this invention is that the 

10 dialogue models may change dynamically as the behavior 

and goals of the customer becomes more evident, the task 
evolves, and the environmental situation changes. 

This invention also allows for the dynamic update of 
the content of each dialogue model 30, based on user 

15 behavior and system performance. For example, a speech 
directed dialogue model 34 may be very effective for 
billing tasks, but is not effective for repair tasks. In 
this case, touch tone model 32 may be adapted to be more 
likely to be selected for nodes related to repair tasks 

2 0 and to include repair options in a more prominent 

location within the interface. This functionality helps 
ensure that each interface type is customer-centric. 

Now referring to FIGURE 2, a flow diagram of a 
communication system for selecting a user interface, 

25 depicted generally at 100, is shown. In operation, a 
particular node or dialogue state is begun within 
communication system 100 at 110. When the node is begun 
112, criteria evaluation module 114 first considers 
criteria 132 stored within criteria library 130. 

30 Criteria 132, in the present embodiment, includes cost 
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134, customer satisfaction 136, automation rate 138, task 
completion rate 140, task complexity 142, confidence of 
outcome 144, time in system 146, list of length 148, VUI 
for previous state 150 and dialogue state counter 152. 
5 In an alternative embodiment, additional criteria may be 
listed within criteria library 130 and some of the 
criteria listed in the present embodiment may not be 
included. 

Criteria library also includes a criteria weight 154 

10 corresponding to each criteria 132. Criteria weight 154 
reflects the relative weight or importance placed on each 
particular criteria. For instance, if an organization is 
attempting to keep costs low, the criteria weight 154 for 
the criteria of cost 134 would be set at a relative high 

15 valuation. In some embodiments, criteria weight 154 may 
be set at 0 for various criteria 132 effectively 
eliminating those criteria from consideration. 

Cost 134 is a reflection of the cost of a particular 
interface type. For example, a service representative 

20 168 is typically the most expensive form of user 

interface while a touch tone interface 166 is typically 
the least expensive form of user interface. 

Customer satisfaction 136 is an indication of the 
degree to which callers, customers, or constituents are 

25 satisfied with the particular user interface. Automation 
rate 138 is an indication of the rate at which user 
interface selection is more for automated systems (such 
as a touch tone or directed dialogue system) versus 
non-automated systems (such as a service representative) . 

30 Task completion rate 140 is the rate of successful 
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completion of tasks by a particular user interface type. 
In a particular embodiment task completion rate 140 may 
be specific to the successful task completion rate of a 
particular user interface type to a particular node or a 
5 particular communication task module (as shown in FIGURE 
1) . 

Confidence of outcome 144 is an indication of the 
likelihood that a particular node will be successfully 
completed. Time in system 146 is a reflection of the 

10 average time a user interface type takes to complete a 
particular node. List length 148 is a node specific 
characteristic reflective of the number of list options 
associated with a particular node. Interface from 
previous state 150 is an indication of the user interface 

15 type that was used by the previous node. Dialogue state 
counter 152 is a counter to keep track of the number of 
times a user has used a particular dialogue state for a 
particular node. The dialogue state counter is intended 
to indicate when a user continues to fail to complete a 

20 task after several attempts using a particular user 
interface type. 

The use of tasks broken down into nodes as well as 
dialogue state counter 152 helps the customer from "going 
backward" in the interaction. "Going backward" occurs 

2 5 when the interface style does not change and the customer 

fails in accomplishing their task with a particular style 
(for whatever reason) , and the customer may have to 
repea t information. In other words, the customer may be 
asked to re-enter information that they previously 

3 0 provided. With this invention, the library provides a 
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dialogue model knowing the current state of information 
which then precludes or avoids prompting the user to re- 
enter previously acquired information. 

Criteria evaluation module 114 retrieves the 
5 criteria 132 and corresponding weights 154 as described 
with respect to criteria valuation module 18 shown in 
FIGURE 1. Next, user interface scoring module 116 
interacts with user interface library 160. User 
interface library 160 includes criteria weight 

10 information for a plurality of user interface types. In 
the present embodiment the user interface types include 
statistical language model 162, directed dialogue model 
164, touch tone model 166, and service representative 
168. In an alternative embodiment, additional user 

15 interface types may be included and less than all the 
current user interface types may be included. The 
present invention encompasses web-based and wireless 
web-based user interface types. 

Criteria scores 170 are particular to each user 

2 0 interface type and a criteria score corresponding to each 

criteria 132 listed within criteria library 130 is 
preferably maintained. Criteria scores 170 are 
preferably a relative reflection of the strength of each 
particular interface type with respect to each criteria 
25 132. For example, with respect to cost 134, statistical 
language module 162 will have a lower criteria score than 
directed dialogue module 164 because a statistical 
language model type interface 162 is typically more 
expensive than directed dialogue type interface 164. 

3 0 However, touch tone type interface 166 will have a higher 
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186, pitch 188, modulation 190, and attitude of 192. For 
each factor persona library 180 also includes persona 
score of 194. In an alternative embodiment, more or 
fewer call specific factors may be included within 
5 persona library 180. In the present embodiment, persona 
consideration module 118 operates similarly to persona 
evaluation module 24 as shown in FIGURE 1 and is 
primarily directed to considering the appropriateness of 
voice activated user interface types. 

10 After the persona consideration module approves of 

the interface selection, the node is advanced to allow 
interaction with user 120. If the node is successfully 
completed, the method is repeated for the next node. If 
the node is not successfully completed 124, criteria 

15 evaluation module 114 reconsiders the criteria 132 

contained within criteria library 130 for the particular 
node 112. In effect, the process is repeated. In 
addition, information related to handling node 112 is 
incorporated into feedback module 128. Feedback module 

20 128 interacts with user interface library 160 to adjust 
criteria scores 170 and with criteria library 130 to 
adjust criteria weights 154 appropriately. 

Now referring to FIGURE 3, a flow diagram showing a 
method for selecting user interface dialogue nodes is 

2 5 shown. The method begins 2 00 by determining the 

appropriate criteria for the first node of a task 212. 
The criteria for the first node of the task are then 
scored 214 and a dialogue module or interface type is 
selected to complete the first node of the task 216. 
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In some embodiments (not expressly shown) , an 
additional step may be implemented following selection 
step 216. This added step would preferably gather feed 
back information pertaining to the results of the 
5 customer 1 s experience with the first node. For example, 
if the customer fails to successfully complete the first 
node or if there is a high background noise, this 
information may be considered for the next step, 218. 
This additional step my then be repeated after each 

10 subsequent selection step 222 and 228. 

Next, a selection criteria for a second node of a 
particular task are determined 218. The selected 
criteria for the second node of the tasks are then 
weighed or scored 22 0 and an appropriate dialogue module 

15 is selected to complete the second node of the particular 
task 222. These steps are then repeated for each ensuing 
node of a task such that for the n th node of a particular 
task the appropriate criteria are determined 224 and then 
scored or weighed 226 to determine and select an 

20 appropriate dialogue task 228. 

The present invention advantageously allows the 
unique advantages of particular dialogue models matched 
with nodes to the customer's and organization's 
advantage. The selection of which dialogue model is a 

2 5 dynamic and customizable approach that can be adapted 

over time based on the priorities defined by the company, 
user behavior, and system performance. 

Now referring to FIGURE 4, a flow diagram of a 
communication system, depicted generally at 200, is 

3 0 shown. After node 2 08 within a communication action is 
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entered, selection process 210 operates to identify and 
select a dialogue model to complete the communication 
action. The selection process utilizes criteria 
evaluation module 212 to communicate with criteria matrix 
5 230. In the present embodiment, criteria evaluation 

module 212 receives a request for a dialogue model from 
node 2 08 and initiates communication with criteria matrix 
230. Criteria matrix 230 is generally operable to 
calculate a dialogue score. After calculation of the 

10 dialogue score, selection module 214 preferably uses the 
calculated dialogue score to select an appropriate 
dialogue type for completion of the communication action 
and the selected dialogue model is preferably used to 
interact with a user 216. A determination is then made 

15 as to whether the selected dialogue model successfully 
completed the communication action 222 or whether the 
communication action was not successfully completed 220. 

In the present embodiment, criteria matrix 230 may 
also be referred to a VUI Criteria Matrix or a dialogue 

20 selection matrix. Criteria matrix 230 includes a 

criteria weight module 236, criteria score module 238, 
dialogue scoring module 240 and dialogue selection table 
242. Criteria weight module 236 is in operative 
communication with interface criteria weight library 232 

25 and may selectively obtain weight values 246 for criteria 
244. Criteria score module 238 is also in operative 
communication with criteria score library 234 and may 
selectively obtain score values 254 for criteria 252. 
Dialogue scoring module 240 uses the criteria weight 
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values 246 and criteria score values 254 to calculate a 
dialogue score . 

In the present embodiment dialogue scoring module 
240 may calculate a dialogue score using the formula: 

where 

DS = dialogue score 
N = criteria type 
W = criteria weight 
10 S criteria score 

X = number of criteria. 

An example operation, single criteria score is 
calculated from an instance where a single criteria type 
is being evaluated such as: 

15 D5 = 2 a=1 (l 1 xl00 1 ) 

In another example evaluation, three criteria types 
are considered in the calculation: 

DS = 3 (25, x 40, ) + (.75 2 x 100 2 ) + (,25 3 x 80 3 ) = 1 05 

In the alternate embodiments, additional criteria 
20 types may be included in the calculation of the dialogue 
score . 

Criteria Matrix 230 further includes dialogue 
selection table 242. Dialogue selection table 242, as 
further described in FIGURE 5, includes dialogue types 
25 corresponding to dialogue score ranges. For a given 

dialogue score, dialogue selection table 242 preferably 
provides a recommended dialogue type . 

In the present embodiment, interface criteria weight 
library 232 stores criteria 244 and corresponding weight 
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valves 246. Criteria score library 234 stores criteria 
252 (which corresponds to criteria 244 stored in criteria 
weight library 232) and corresponding criteria scores 
254. In an alternate embodiment, criteria weights 246 
5 and criteria scores 254 may be stored within a single 
library or database which may be incorporated into 
criteria matrix 230. Further, criteria types 244 listed 
in FIGURE 4 include cost, customer satisfaction, 
automation rate, and task completion rate, which also 

10 correspond to criteria types 1-4 listed in criteria score 
library 234. Alternate embodiments may include more or 
fewer criteria types as appropriate for calculating a 
dialogue score. 

Criteria matrix 230 further includes feedback module 

15 260. Feedback module 200 establishes whether a selected 
dialogue type has successfully completed a communication 
action. Feedback module 260 adjust criteria weights 246, 
criteria scores 254, or dialogue selection table 242 to 
achieve defined performance and business goals. Criteria 

20 weights 246, criteria scores 254 and dialogue selection 
table 242 may also be manually adjusted by a system 
administrator as performance and business needs dictate. 

Now referring to FIGURE 5, a dialogue selection 
table, indicated generally at 242, is shown. Dialogue 

25 selection table 242 includes a dialogue score column 260 
listing multiple dialogue score ranges. Dialogue 
selection table 202 also includes dialogue path column 
262 which includes multiple dialogue types corresponding 
to dialogue score ranges 284. 
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Although the disclosed embodiments have been 
described in detail, it should be understood that various 
changes, substitutions, and alterations can be made to 
the embodiments without departing from their spirit and 
5 scope . 
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